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COUNSELING RESEARCH SHOULD BE DESIGNED TO DISCOVER 
IMPROVED WAYS OF HELPING CLIENTS PREVENT OR SOLVE THEIR 
PROBLEMS. THE IMPLICIT ASSUMPTION THAT COUNSELORS ALREADY 
KNOW HOW TO counsel SHOULD BE QUESTIONED. THE DEPENDENT 
VARIABLE SHOULD BE CLIENT, NOT COUNSELOR BEHAVIOR. A NARROW 
OUTLOOK OF COUNSELOR PURPOSES AND METHODS MAY BE DANGEROUS. 
COUNSELING RESEARCH SHOULD BE DESIGNED SO THAT DIFFERENT 
POSSIBLE OUTCOMES LEAD TO DIFFERENT COUNSELING PRACTICES. 
RESEARCH SHOULD GAIN THE RESPECT OF OTHER DISCIPLINES. THE 
TEST OF RELEVANCE SHOULD BE APPLIED IN PLANNING COUNSELING 
RESEARCH. OUTCOME CRITERIA OF COUNSELING RESEARCH SHOULD BE 
TAILORED TO THE BEHAVIOR CHANGES DESIRED BY THE CLIENTS AND 

counselors involved, the criterion for each client should be 

APPROPRIATE: GROSS CRITERION MEASURES, SUCH AS GRADE POINT 
AVERAGE, HAVE DANGEROUS WEAKNESSES. CRITERION BEHAVIOR 
EXTERNAL TO THE COUNSELING SITUATION ITSELF IS IRRELEVANT. 
THERE ARE SEVEN POSSIBLE FRUITFUL AREAS FOR COUNSELING 
RESEARCH— (1) ESTABLISHING THE COUNSELOR AS AN EFFECTIVE 
AGENT OF CHANGE THROUGH ALTERNATIVE WAYS, (2) LEARNING THE 
SKILLS OF BUILDING HUMAN RELATIONSHIPS, (3) HELPING MEMBERS 
OF SPECIFIC SUBPOPULATIONS THROUGH MORE EFFECTIVE PROCEDURES, 
( 4 ) EXTRAPOLATING FROM RESEARCH IN OTHER DISCIPLINES, (5) 
BUILDING A LIBRARY OF EFFECTIVE MODELS, (6) IMPROVING THE 
CAREER DECISION PROCESS, AND (7) PREVENTING PROBLEMS. (WR) 
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Future Directions for Counseling Research^ 

John D. Knnbolts 
Stanford University 

A little over a year ago because of some family connecti^#; I attended 
the annual convention of the American Dental Association in Francisco. 
While waiting for the family to reassemble one afternoon, 1 b^pyaa%d to be 
seated outside one of the meeting rooms and overheard a conw#vsa^loo which 
seems now to have particular relevance for research in counse^lii|g» I shall 
try to reproduce the conversation as nearly as I can remember it while 
admitting fully that perceptual distortions can occur. The t<^«taation 
went something like this: 

"I pissed that last session, but I heard some of the fellows muttering 
on the way out. Is it true that research has proved debl^istr are 
unnecessary?” 

"Just that they make no significant difference.” 

"How did they prove that?” 

"Had five dentists cooperating. Out of 20 patients ealli.ng each . 
dentist, ten picked at random were asked to wait six moh^s% The 
other ten were given appointments innediateiy." 

"So what happened then?" 

"A year later both groups were examined, and both groups ^a4 sT>out 
the same average number of teeth, about the same averag<i antnisl income, 
and were about equally happy when asked to respond on • S^en^’pCint 
rating scale." 

^ Paper presented at the Invitational Conference on the Asseasaieflt of 
Counseling Outcomes^ St Louis, Missouri, Washington Unlveraiv;y^ 

January 12,. 1967 . 
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"Weren't there any differences?" 

"There was almost a significant difference on one rating scale." 

"What one?" 

"Degree of respect toward dentistry as a profession." 

"Well, at least there's a trend in our favor." 

"Nope, control group had more respect." 

"Now wait a minute. This kind of research is ridiculous. You can't 
tell me dentists don't do any good. Why, I know a little girl with 
a cleft palate whose speech could scarcely be understood. She was 
shy and withdrawn^ teased by her playmates, and some of her teachers 
thou^t she was mentally retarded. Then a. dental surgeon produced a 
prosthesis to close the gap in the soft palate and enabled her to 
apeak normally. You should hear this girl tell what a tremendous 
difference it made iu her life." 

"Significant at what percent level? That's only testimonial evidence. 
No respectable Journal would publish anecdotes like that in this 
scientific age." 

"You think this fancy research project is pretty good, don't you?" 

"Well, it's going to be published." 

"there is one fundamental thing wrong with it." 

"What's that?" 

"It makes no significant difference." 

"What do you mean?" 

"It makes no difference that the results are negative, and it would make 
no difference if the results had been positive. Furthermore, it was 
obvious from the beginning that a study like this could not possibly 
advance dental science one bit, no matter how it came out." 

"You better <»xplain yourself." 

"OK, vdiat are the dentists who heard about these negative results 
going to do differently from now on?" 

"Nothing. They complained about the study- "Efmall N, irrelevant criteria 
(didn' t even count the number of dental caries) , need for a longer 
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follow-up interval, all that sort of thing. And besides they thought 
it didn't apply to them." 

"Any of them threaten to quit dental practice because of this evidence?" 
"Of course not." 

"Now what do you suppose dentists would do differently if the results 
had been favorable?" 

"Nothing different. But they would probably feel more self-satisfied 
about xdiat they were already doing." 

"Precisely. No matter \diich way the study came out, dentists would 
continue to do just what they've always done. We will never advance 
the dental profession by doing that kind of research." 

"What kind of research do you think we should do?" 

"Research that makes a difference in the effectiveness of what dentists 
do. We need better ways to remedy dental defects, help people care for 
their own teeth, and prevent dental problems." 

''That's pretty general. Can you give any specific research problems 
which might affect what dentists do?" 

"Sure. We need to know the optimum ratio of mercury to silver alloy 
in amalgiim to achieve various objectives--hardness, adhesiveness, 
absence of discoloration. We need to know the effect of diet bn tooth 
permanence. We need to kiiow how to prevent the chemical imbalances in 
saliva that cause enamel dissolution or the precipitation of calculus. 

We need to know . . ." 

"OK, OK. But what if we solve all these problems you were about to 
list? What would dentists do if all our preventive and remedial efforts 
were successful? We would be out of work." 

"That is precisely the objective of the dental profession— to eliminate 
oral disease and thus eliminate the need for the current type of costly 
dental services. If we could ever solve all the problems of oral 
disease, we could turn our efforts toward new problems. Any true 
profession tries to reduce the need for its own services by preventing 
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or solving more efficiently the problems for which it assumes responsi* 
bility." 

That was the conversation that I heard, or thought I heard, or perhaps 
dreamt. In any event, it started me thinking about the kind of research 
that I have been doing. How much of the research which I have done has 
actually made any difference for anyone's practice? For that matter, what 
difference has it made in terms of my own practice of counseling or teaching? 

I am forced to confess that a great deal of my own research has made absolutely 
no difference to anyone. It seems obvious now that I should have know so 
in the beginning. 

As a result of this introspection and a review of counseling research, 
particularly that of the last three years, I have arrived at some conclusions 
which I shall present to you in the form of three propositions. 

Proposition: I ; Counseling Research Should Be Designed to Discover 

Improved Ways of Helping Clients Prevent or Learn 
to Solve Their Problems . 

The Implicit Assumption That We Already Know How to Counsel 

Perhaps the most disturbing note running through the counseling litera- 
ture is the implicit assumption that counselors already know the best way 
to be of assistance to each client. Many of the research studies we see 
attempt to evaluate counseling but give very little attention to exactly 
^at it is that counselors dp. Everyone seems to know implicitly what 
counselors dO; If so, then the next step is to prove that what they do 
is good for something. The danger in this belief is that ir may cause us 
tp stop searching for new ways to help clients. If we are already coh- 
vinced that we. know the best way to help clients, the next job is to conduct 
research whicli will attempt to prove that "counseling" works better than 
nothing. Such research is futile no nmtter what the results; 

When research efforts produce no significant differences between 
counseled and non-counseled subjects, we have a ready set of rationalizations. 
Perhaps counseling has long term effects. If only we could conduct a long 



t 



♦'i' 

f 



range follow-up, we would find greater effects. Criterion instruments can 
usually be blamed for part of the difficulty, and of course a larger N is 
always desirable. 

Every now and then (Could it be about one time in twenty?) we do see 
studies which show significant differences. However, I do not wish to argue 
here whether counseling does or does not have any effect on clients. I 
think we are wasting our time even to ask if counseling has an effect. I 
personally am convinced on the basis of private evidence that couns^.ling does 
have some effect on some clients for some kinds of behavior change. I 'am 
not interested in proving to the world that some generality knoxm as counsel- 
ing is generally effective for general clients. What I want to know is how 
counselors can do an even better job of helping their clients. It will not 
do us any good at all to find out that counseling as it is presently con- 
stituted is slightly better than nothing in helping clients. I am afraid 
that if we did receive some conclusive evidence of counseling's value, all 
efforts to improve the counseling process would cease. 

In order to generate appropriate kinds of research problems, we need 
to begin xd.th the problems of our clients. We need to ask for each client, 
'■'What is the problem that he ‘ is facing? What kind of behavior must he 
engage in if he is to resolve that problem? What are some of the alternative 
actions that the counselor can take that would help him to engage in that 
kind of behavior?" 

The behavior changes desired by each individual client and his counselor 
constitute the foundation upon which our teae&xsh must be based. To - 
do anything else is to attempt to produce changes which are not necessarily 
desired by either the client or the counselor. But it is essential before 
research begins that we eliminate the idea that we already know how best 
to counsel. Instead, we need to consider some alternative ways of helping 
our clients attain their life desires. 

Dependent Variable As Client, Hot Counselor, Behavior 

It seems almost too obvious to mention, but the target of counseling 
research ought to be the behavior of the client, not the behavior of the 

- 5 - 



o 



counselor. Some technically excellent work has been done on the effect of 
alternative types of client behavior on the responses of the counselor. It 
may be of some general interest to know that when a client is hostile toward 
a counselor, the counselor emits certain types of anxiety responses. However, 
this would be useful information only if we were interested in training clients 
to make counselors anxious. A far more valuable research topic would be to 
determine what types of counselor responses make the client less anxious. 

The idea of using coached clients to investigate counselor responses is 
deceptively attractive. It is attractive because of the ease of establishing 
rigorous experimental designs which may produce significant results. The 
deception is in the fact that the results do not actually advance the 
counselor's knowledge of what actions he may take to help his clients. 

Another type of study that appears somewhat irrelevant to the client's 
welfare investigates the counselor's ability to predict certain patterns of 
client speech or other behavior. The purpose of counseling is not to predict 
what the cUent will say. Besides, Carkhuff's (1966) review of the evidence 
showed that increasing Amounts of graduate training lower the ability to 
predict client behavior. If we really want to help people predict other 
pepple*s behavior, we ought to be teaching that skill to the client, not 
just the counselor. However, it is apparent tha«. we do not yet know how 
to predict other people's behavior any better than any other normally 
intuitive person. When we do learn how, we should list it as a skill some 
clients may wish to learn. Our research should testify that we work for the 
welfare of the client, not the counselor. 

The Danger of a Narrow Outlook 

Host modern industries in America are no longer tied to a single product 
or a single process. A company which manufactured buggy whips in 1890 might 
have been very prosperous at the time and even for several years thereafter, 
but if the company conceived of itself as being in the buggy whip manufacture 
ihg business, it was doomed to failure. It would have survived if it had 
conceived of its role as being in the transportation business. A far-sighted 






buggy whip manufacturer would have said, *^e are in the transportation business 
and therefore it is our job to manufacture those articles \^ich enable people 
and goods to be transported as safely, quickly, and economically as possible." 
Thus, the advent of the automobile would be viewed as an opportunity to expand 
the product line rather than as a threat to financial security."^ ; 

Right now we see the beginnings of a similar development in the^bpok 
publishing business. Some book publishers are going to say, "We are in the 
book publishing business, our job is to manufacture books^ and we view all' . 
other means of storing and transmitting words and images with hostility and 
suspicion." But far-sighted book companies are going to view themselves 
more broadly as part of the communications industry. Thp^ will ask how they 
can transmit pictorial and verbal images to the people who wish them most 
expeditiously. One example of thi a broader outlook is represented in the 
merger of the ilmerican Book Company with Litton Industries.. The development 
of computers, copying machines, and random access microfilm storage files 
foretells a fundamental revolution in what used. <tp be the publishing industry, 
but is how the citeunications industry. 

We are faced with a similar danger, or opportunity, in the field of 
counseling. In the face of innovations we inay be tempted to reply, "We are 
in the counseling profession. Our job is to counsel. We must provide one 
counselor vbo will be available to talk at least X number of hours each year 
with each individual." We must not take such a narrow, view of Our function. 

We would be much more likely to promote the welfare of clients, counselors 
and citizens if we were to view ourselves as part of the’ e^^ccation industry. 

It is our job to help our clients learn. For a variety of reasons ■ 
many of them have failed to learn some of the basic things which enable them 
to function in society. Our job is to promote, somehow, the learning that 
jDOUst be accomplished. Right, now we have some ideas how it might be accom- 
plished through a process called counseling. Perhaps someday sOmeone will 
develop improved ways for helping to proinbte this new learning. When better 
ways are developed, far-sighted counselors will eagerly test them, Sdopt 
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thra if they^ work, and continue to improve them* 

Counselors are in the businessof prompting behavior changes along certain 
dimensions Just as classroom teachers are in the business of promoting behavior 
changes along certain other dimensions. We go by many different titles 
professors, teachers, counselors, psychologists, administrators, psychptherar 
pists, and even psychiatrists, but fundamentally we are all in the seme 
business, that of helping our clients learn those things necessary for their* 
6\m welfare. Our research must Improve the process. 

Proposition II : Counseli ng Research Should Be Designed So That 

Different Possible Outcomes Lead to D ifferent 
Counseling Practice. 

When I was a young and naive graduate student at the University of 
Minhesbta, I soon became duly, impressed with the importartce of using the 
proper experimental design, the right statistic for the necessary assumptions, 
the need for properly validated measurement instruments, and the necessity of 
choosing a representative sample. So much effort was involved in learnings 
these aspects of research that it was not until later that I really appreci- 
ated the far more important lesson from my graduate training, namely: If 

you ask the wrong question, no amount of statistical sophistication cati .make 
the results valuable. Let’s consider some of the factors which keep us from 
asking the "right" questions and how we might increase our ability to ask 
questions which will make a difference. 

Reseatch Which Gains the Respect of Other Disciplines 

Sometimes we read surveys which report that counseling psychology ranks 
toward the bottom of the prestige hierarchy within the field of psychologyi 
Some of us have felt tangible effects of being at the bottom of the pecking 
order in the psychological community. We are bothered by this and sometimes 
protest (iistially to other counseling psychologists) that we are entitled to 
just as much reppect as any other type of psychologist. But tespect cannot 
be requested; it cannot be demanded; and it cannot; even be legislated. Respect 
is earned, and it is earned by demonstrated competency in mastering the 
in one's own field. 
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of some of the academic disciplines. Much of this research is knowledge 
for its own sake" (o.r facts gathered for no known reason). (Actually t of 
course^ good researchers* ■■ io other disciplines gather facts for very good 
reasons, usually to provide evidence to support or disconfirm a possibly 
useful*, theory.) It is a temptation to adopt the apparent policy of those 
who have more prestige than we have and therefore to conduct research which 
has ho known practical benefit. Much of the research which I read in our 
professional journals appears to be just of this type, collections of facts 
(seldom even interesting facts) which make no possible difference in the 
practice of counseling. 

It is not our job to ainass knowledge for its own sake. Our job is to 
improve the practice of counseling for the welfare of our clients. The facts 
we collect should be those which make some kind of a difference in what coun- 
selors do* We will gain the respect of other disciplines when we put our own 
house in order and accomplish the purposes for which we work. We will hot 
gain the respect of other disciplines by becoming a superficial imitation of 
a basic science while at the same time counseling practice and cpunseling 

r6S68Vch bGflr tio. rGlfltionship to G8ch othGr* 

New junior colleges are being established and sone times they begin by 
modeling themselves after soine prestigeful university. Although the primary 
purpose of the junior college is to provide adult and terminal education 
for many students who would not benefit from a traditional curriculum, policies 
and courses are often instituted in an attempt to model the high prestige 
institution. High Standards are set in irrelevant courses, many students 
fail, and for some strange reason this high failure rate is viewed with pride. 
The basic difficulty is that an inappropriate model had been chosen. The 
junior colleges which are gaining the most respect are those which are 
accomplishing their own purposes most effectively. Those which are providing 
meaningful courses^ stimulating teachers, and relevant occupational training 
have won the respect, not only of the general public, but of responsible 
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professional persons at the major universities also. Demonstrated competence 
In accomplishing one' s own purposes Is the route to attaining respect from . 
others. 

Counseling research wf.ll begin to earn the respect It deserves when the 
results of that research begin to Improve counselors' abilities to help their 
clients. 

Nothing that I have said up to this point should be Interpreted to niean 
that I op;K>se the seeking of knowledge for Its Own sake. Many scTShtlfic 
advances and technological Improvements have occurred unexpectedly from 
scientific attempts merely to achieve a better understanding of some phendm' 
enon. Such fundamental research will continue, should continue, and indeed 
must continue. Counseling psychologists have an obligation to keep In 
contact with that basic research which may possibly have relevance for develop- 
ments in the field of counseling. If, from time to time, the counseling 
psychologist should wish to contribute to this fundamental research, no diie 
ought to object. However, if we make oiir primary research activity the devel®P“ 
ment of knowledge unrelated to the Improvement Of counseling practice, then 
X think we are neglSctlng the first responsibility of counseling researchers^ 
The first order of business is finding out what x^e need to knOw in order to 
Improve the practice of counseling. 

The Test of Relevance in Planning Research 

C. H. Patterson (1966) has xnrltten, "To be engaged In research is dlinost 
an end In Itself, and one can actually continue^ sometlines for years, doing 
research without having to show what he has accomplished, or to demonstrate 
the value of any results" (p. 130). I agree strongly with Patterson that 
such trivial research has many unfortunate consequences for the entire 
profession. The solution that I see, however, is not to give more importance 
to theory and discussion but to Increase the relevance of the research that 
we do. 

To this end I would suggest that in the planning stage of every doctoral 
dissertation and research proposal In the field of counseling the test of 
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relevance be The test of relevance consists of asking one simple 

question and probing the answers: What will <jg 41tfgreBtlV i t 

Suppose a doctoral candidate proposes a research study on some probl^ 
in test interpretation. He wonders how accurately parents can estimate the 
relative test results of their own children. He has developed a whole series 
Of hypotheses that parents will be aMe to estimate certain abilities more 
accurately than other abilities, certain personality traits more accurately 
than other personality traits. Let us now apply the test of relevance. 

First question: "Suppose you find that parents are able to estimate their 

own children* s verbal ability significantly better than they are able to 
estimate their own children’s mathematical ability, what would counselors 
do differently as a result?" Second question: "Now suppose that you find 

that parents are just as able to predict their own child’s mathematical ability 
as they are to estimate verbal ability, what would counselors do differently 
as a result of that finding?" If it turns out, as I expect it would, that 
Couhselprs would do just exactly the same thing regardless of which way the 
results came out, then that proposed research is not really relevant to the 
field of counseling psychology. It may be a contribution to some type of 
basic, knowledge, and it may be a publishable research study. But if we allow 
that research to be conducted, we will have lost one more opportunity to 
improve the practice of counseling, and we will have taught one more student 
that research is: merely academic calesthenics and need not contribute to 
solving real prpblons. A job that is not worth doing at all is not worth 
doing well . 

But the test of relevance can be taken one more step to make it a 
help in constructing new research problems, not merely a way of eliminating 
irrelevant proposals. The extension consists pf asking what possible alterna- 
tive counselor actions might be taken as a result of having the proposed 
additipnal infornmtion. Let’s consider the research study done by MacOuiddy 
(1964) concerned with a problem of test interpretation. MacQuiddy, in 
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essence, asked the question: I£ counselors know how accurately each parent 

had estimated each ability of his child, would he interpret the results of 
the testing any differently to the parents? More specifically, if a counselor 
knows that a parent has estimated some of his child's abilities quite 
accurately but is markedly wrong in estimating certain others, and assuming 
that the counselor would like the parent to accept the results of the testing, 
should the counselor begin by interpreting first the score that the parent 
has estimated most accurately, or should he begin by interpreting the most 
discrepant test score? MacOuiddy arranged for several school psychologists 
to Interpret to parents their child's test results after first obtaining 
the parents' own estimates of their child's abilities e MacQuiddy's hyphtbesis 
was that parents would accept the results of the testing more readily if 
counselors began by Interpreting, the score \diich the parent had, estimated 
most closely, then interpreting the scores that were increasingly discrepant 
until the last score interpreted was the score that the parent had missed 
by the greatest amount. The results were surprising, for they came out in 
just the Opposite direction from that hypothesized. The parents who accepted 
the results most thoroughly were those in the group which received the most 
discrepant score first. Those ^ose judgment was initially confirmed were 
least willing to accept the test results. 

This study is cited, not as an example of an earthshaking experiment, 
but aa an exmnple of a study vdiich can affect the practice of counseling 
in some small way. If I were a counselor interpreting a group of test 
scores to parents, and if I had some prior knowledge of what the parents 
thought their children's abilities were, and if I wanted the parents to 
accept the results of these standardized tests ^ then I would begin by telling 
each parent about the test result that was most different from what the parent 
expected i In short, I would do something differently as a result of knowing 
the results of MacQuiddy's research study. 

Now, you may say that I would be making a mistake to base my practice 
upon one study done by one person in one school system on a Small sample 
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using only certain tests In one year. Obviously, the results of this study 
need to be replicated. But the basic point Is that If this study Is repli- 
cated, and If the results do come out the same way In these replications, 
then we have some firm knowledge about how counselors could act In order to 
bring about a given result. 

One other Important point: The rationale for MacOulddy's study was 

based on Festlnger's cognitive dissonance theory and related basic research. 
The results of MacCuiddy's research are relevant to the theory of cognitive 
dissonance and may contribute to the support or dlsconflrmatlon of certain 
propositions In that theory. Practical research questions should not be 
Isolated from basic research knowledge or fundamental psychological theories 
but. Indeed, should be relevant to them, and furthermore, should contribute 
to such theories as a by-product. 

The division of labor that I am proposing Is that we counseling psycholp’' 
gists use the theories and basic research of other disciplines to formulate 
questions, the answers to which affect counseling practice; we leave to 
Festlnger and other theorists the problem of Integrating the results of 
practical research in the modification Of their theories. 

fronnattlon 111 ; Outcome C r iteria of Counseling Research-Shovl^-Bs 

Ta^^ored to the Behavior Changes 
Cl and Couns elors Involved. 

Blsewhere I have argued that the goals of any counseling should be 
formulated in terms of observable behavior changes desired by both the client 
and the counselor (Krumboltz, 1966a, 1966b), I shall not reiterate the 
reasons why I consider such a formulation desirable, but the essential point 
here is that the goals of different counseling sessions may vary nmrkedly. 
one client may wish to reduce the anxiety he experiences in connection with 
taking tests. Another may be concerned about his ab^ity to form friend- 
ships. Still another may have inadequate study habits. The goals of each 
counseling session are different for different clients, and therefore it is 
impossible to apply a single criterion to evaluate counseling in its totality. 
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Appropriateness of the Criterion for Each Client 

If we examine some of the criterion measures which have been used to 
evaluate counseling in recent research studies, we are impressed by the 
creativity involved in generating so many possible desirable outcomes of 
counseling. But was each possible outcome desired by the client who came 
to the counselor in the first place? 

For example, one criterion measure is the number of constructs borrowed 
from the therapist. Did the client come to the counselor in order to learn 
to use the same terminology as the counselor's? Would the client be happy 
to learn that success had occurred in his case because he was now using more 
of the terminolo^ of the therapist than he did in the beginning? 

Another criterion TOasure is an enlarged time perspective. Did the 
client come to the counselor because he was dissatisfied with his narrow 
time perspective? Would he be happy to know that counseling had been judged 
successful with him because his time perspective had been enlarged? 

Another criterion measure is the; amount Of participation in groups. If 
the client had come to the Counselor because of excessive loneliness and 
inability to communicate with other people, and if he and the counselor agreed 
that it ix>uld be desirable for him to learn how to get along with other people 
in group situations, then I think an excellent criterion for him would be his 
amount of participation in groups. If, on the other hand, he came to the 
counselor because of an inability to manage his own time and poor study 
habits, then an increase in the amount of time he spent in groups might be a 
sign that the counseling had been a distinct failure. 

Uniea# the criterion n^asure represents the kind of behavior change 
desiri^ by the client arid the counselor in each individual instance, the 
coiihseiing of that client can not be evaluated. This means that any counsel- 
ing butcpine research with a group of clients must be undertaken with clients 
all of whom desire to make the same type of behavior change. 

The Danger in Gross Criterion Measures 

One of the favorite criterion measures in educational and counseling 
research is the ubiquitous grade point average (GPA) . It seems like a 
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desirable criterion measure since almost everyone ^ 70 uld like to have a higher 
GPA. A well designed study by Winkler, Teigland, Hunger, and Kranzler (1963) 
illustrates the difficulty. The subjects were 121 fourth*graders, all of 
vThom had a conmon problem in that they w^re "underachievers." They were 
randomly assigned to cl lent -centered individual counseling, cl lent- centered 
group counseling, a reading instruction group, a Hawthorne control group, 
and a no- treatment control group. The criterion measure was the amount of 
increase in the GPA. As you might expect, no statistically significant 
differences were found among these five groups, but the group that tended to 
show the largest increase in GPA was the no- treatment control group. 

What is wrong with Using a gross measure like GPA as a criterion measure 
in counseling? 

1. GPA is dependent upon innimrerable factors beyond the control of the 
treatment procedures. The student's inherited characteristics, his home 
environment, his years of education, and the already established expectancies 
of four or five teachers are all beyond the control of the counselor. GPA 
is a very difficult measure to alter under the best of circumstances. To 
expect a fen7 hours of talk with another person to alter radically such an 
insensitive criterion measure is to be most unrealistic. 

2. When grades are assigned "on the Curve" (and they usually are even 
xdien the stated policy is that they are not) , any improvement in one person' s 
GPA means that someone else's GPA is probably going to decline. If an 

L entire group is given some kind of effective treatment, the resulting imprOve- 

I ment T'Till- not be reflected in the GPA because the teachers will still be 
distributing their grades in about the same proportions. The average will 
remain approximately the same. 

3. ^e time required for the experimental counseling treatment sub- 

I" ^ . 

I' tracts from the time available for regular classroom instruction or study. 

I In order for counseling to show any ef^ .t, it must be shown to be more 

I valuable than an equal amount of time spent with the teacher doing the work 

I which contributes directly to the grade point average. Thus, we are in effect 
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expecting teachers to give higher grades to students T^ho have been rraoved 
from their classes than for students v?ho have remained In their classes an<l 
done the assigned work at the time It was expected. Under these clrcum” 
stances then. It should not be surprising to find that the so-called no- 
treatment control group might very well show the greatest Improvement In 
grade point average. 

Grade point average Is only the most frequently used of gross criterion- 
measures, but there are others, particularly those used In long range follow- 
up studies, x^lch suffer from some of the same disadvantages. Here are a 
fex7 exainples: (1) annual average Income, (2) rating by employer, and 
(3) self-rating of job satisfaction. All are. dependent upon such a multitude 
of other factors that \qe should not be disappointed to find that the few 
hours we spend with a client fall to affect these measures years later* My 
basic point is that progress will be faster if we. avoid using criterion 
measures of such a gross nature. Instead we should confine ourselves now 
to criterion measures which represent the first steps toward outcomes desired 
by the clients. Outcome criteria must not be trivial but oust be an indica- 
tion that progress Is being made tox^ard vital client goals. 

' Criterion Behavior External to the Counseling Situation Itself 

If we are to achieve any valuable assessment of counseling outcomes, 
we must determine whether behavior learned in the presence of a counselor is 
later evidenced in the absence of that counselor. 1 am not at all Impressed 
by evidence that self-references In later coxinsellng Interviews are more 
positive than' earlier self-references. It Is too easy for the counselor to 
selectively reinforce certain kinds of statements and, knowingly or notj 
Increase the frequency of certain types of verbal statements xd.thln the 
Interview Itself. Truax (1966) has shoxm that even Carl Rogers himself 
selectively reinforces certain categories of verbal behavior and not others 
and thus affects their relative frequency in later Intenrlews. Now we 
need to know whether reinforcing certain types of positive self-references 
affects relevant behavior outside the counseling session. 
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Some of the research vje have done at Stanford University has attempted 
to evaluate the amount of career exploration activity that takes place 
during a three'week period of time immediately after the experimental coun- 
seling treatments. It is quite difficult to collect this kind of evidence 
since independent interviewers must be hired to interviex <7 students individually^ 
asking them buffer questions, as well as the relevant questions, to disguise 
the purpose of the interviev;. Then, to determine whether the student reports 
of their exploratory activities are accurate^ a random sample must be 
investigated by some small scale detective wrk (Krumboltz. and Schroeder, 

1965; Krumboltz and Thoresen, 1964). There are many difficulties with this 
criterion measure, but it is a type of behavior that is relevant to the 
reasons counseling was requested in the first place, and it is a behavior 
that occurs after the counseling itself has been completed. 

Some Possibly Fruitful Areas for Counseling Research 
While I certainly will not be all inclusive and probably will not agree 
with myself a year from now, I would like to suggest seven possible areas 
where counseling: research might be profitable at the present time. 

1. Alternative Ways of Establishing the Counselor as Effective Agent of Change 
We have some evidence that the client's expectation that the counselor 
is going to be effective in helping him is related to more effective counsel- 
ing. What is it that establishes the counselor as an effective and competent 
person to the client? Up to this point counselors have relied upon establish- 
ing a warm, permissive, nonjud^ental, understanding relationship as the 
sine qua non of counseling. When; such a relationship is established, the 
client values the counselor most highly (cathexis or transference would, have 
occurred in psychoanalytic terminology) , and the counselor is in a position 
of considerable influence In the client's life. At that point, whether he 
wants it or not, the counselor becomes a much more poi^erful reinforcing agent 
to the client. Even ndnimal cues of approval or indifference become very 
important to the client and can influence the client's actions. Thus, 
establishing a warm and understanding relationship is one way of increasing 
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the pwer of the counselor as an agent of change. But are there other ways? 

To vThat extent would it be Important for the client to be made aware of 
the competence of the counselor in skills respected by the client? Would 
the counselor be more effective if the client knew that the counselor had 
already achieved a high degree of professional success? Would the counselor 
be more effective if the client were told that this counselor had had great 
success with problems similar to his ovm? 

Arnold Lazarus tells of the remarkable success he had in counseling a 
juvenile delinquent referred by the court, a young man who was hostile and 
contemptuous of all Lazarus' attempts to establish a warm and understanding 
relationship. Progress began after a series of events provoked a fist fight, 
during which Lazarus modestly claims he luckily landed a telling blow which 
threw the youth against a file cabinet, dra\d.ng blood and temporarily stunning 
him. The first itovds spoken by the young delinquent after this encounter 
were, "Say, you throw a mighty right for a pen-pusher." This demonstration 
of pugilistic competency established Lazarus as a person highly valued by 
that youngster and made possible a radical transformation in this young man's 
behavior. Moral: There may be more than one way to establish a helping 

relationship. 

Industrial concerns employ people competent in unrelated but prestigeful 
fields to give testimonials about their products; religious organizations 
employ successful athletes to spread the word among adolescents who respect 
athletic success more than theology; and so it seems conceivable that coun- 
selors known for success in areas respected by their clients may be more 
effective than those Of equal ability who are not known for these superficially 
irrelevant competencies. Research on this issue could affect the way coun- 
selors are introduced to clients and the extent to \diich counselors reveal 
information about themselves. 

Though it may grate on the sensitivity of some humble counselors, it 
may be worth asking whether some outward displays of prestige may notenharice 
the counselor's effectiveness. The clients of other professions seem impressed 
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by outward displays of prosperity, such as thick carpets oh the floor, 
expensive furniture, spacious office and reception rooms with gracious 
secretaries. Professional degrees, diplomas and certificates displayed on 
the wall suggest to the client that the counselor is hot one who arrived at 
his position without gaining the respect of some relevant groups. 

Perhaps there are still other ways of establishing the counselor as ah 
effective agent of change. Additional ways could be profitably explored. 

We have m de one feeble attempt to investigate some factors which might 
affect the counselor's effectiveness (Krumboltz, Varenhorst, and Thoreaeh, 
in press) . However, the study was confined to experimentally ohinipulatipg 
the apparent prestige and competency only of a model counselor presented 
on a video tape. No effect was apparent on students Observing, the tape. 
However, the effect of different competency levels and prestige levels of 
the client's own counselor could be a profitable area of investigation. 

2. Learning the Skills of Building Human Relationships 

One fundamental problem presented by a large number of clients concerns 
their ineffectiveness at developing friendships and V7arm human relationships 
of mutual trust. A traditional counselor approach to this problem haa been 
to establish the counselor as the client's friend. If done successfully, 
the client is able to confide to the counselor things about himself that 
ha is unable to coinmuhicate to any other human being. The counselor becomes 
an extremely important person in this client's life, and undoubtedly the 

-If 

experience of being so important to a client is extremely reinforcing to 
many counselors. 

But the fundamental question is xdiether having such a relationship with 
a counselor really enables the client to learn skills in developing sound 
human relationships with other people. If the counselor is not careful, 
the client may learn nothing about establishing solid relationships with 
other people. Worse than that, he may learn some patterns of behavior that 
actually prevent him from establishing such relationships: (1) He may learn 
that the way to establish a good relationship is to do all the talking. 
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expecting the other person to do all the listening; (2) He may learn that 
his ovm personal problems are of overwhelming interest to other people 
vdiile other people' s problems do not seem worthy of discussion; (3) He, may 
learn to talk about his own personal feelings without ever taking any 
constructive action to do anything about them; (4) If he does adopt the 
counselor as a model) he may think that the only way to help other pepple 
is to listen empathically to their problems. It is highly likely that, 
all of these patterns of behavior, if transferred to his relationships with 
other people, may actually impede the establishment of sound relationships 
xdLth other people. ^ 

We need to find put the skills and behaviors xflfhich enable human beings 
to establish ‘dose and satisfying relationships with each other. Then we 
need to discover ways to teach our clients to evidence these skiils and 
behavior patterns so that they themselves can enjoy the satisfaction that 
comes with having a really good friend. Establishing such a relationship 
with the covuiselor himself could possibly, facilitate this learning if handled 
properly, but such a relationship by itself should' hot be considered a 
desirable outcome or necessarily the. only or best way that such skills can 
be learned. We need to examine much more clearly the ways In tdiich solid 
friendships are developed, and we must take pains to teach clients how to 
develop solid relationships so that there is a mutual giving, and receiving 
and hot the one~ sided relationship tbst so frequently develops in the 
traditional counseling interview. 

3. More Effective Procedures for Helping Members of Specific Subpopulatiohs 
It appears that counselors are renmrkably ineffective in dealing with 
certain subpppulations with whom society expects us to have some positive 
effect. The subpopulation known as "the culturally disadvantaged" seems 
remarkably immune to the traditional approaches employed by middle class 
counselors. The Education and Training Committee of. Division 17 has been 
investigating this problem and has recommended that counseling psychologists 
give their at: tent ion to some of the research prbblems involved in wrking 
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more effectively ^rith the culturally disadvantaged. An excellent article 
by Calia (1S66) outlines some of the problems and some possible approaches 

to them. 

Another subpopulation for which we seem ill-prepared consists of 
elementary school pupils. In the last session of Congress the establlstaent 
of the child development specialist was proposed. While the bill failed 
in this session of Congress, its mere existence is some testimony to the 
fact that whatever it is that counselors attempt to do, they apparently are 
not doing it very well in the elementary school. It is quite possible that 
the behavior changes for which counselors work at the secondary school level 
are totally inappropriate at the elementary school level. Research efforts 
heed to establish the type of behavior change problems presented by elemea- 
tary school age pupils and to investigate appropriate procedures for helping 
the children overcome these problems. Additional investigation is needed to 
detepaine teaching procedures, curriculum innovations, as x«ll as counseling 
techniques which will enable these youngsters to unlearn their maladaptive 
behavior patterns and relearn more appropriate ways of coping with their 
envirotmient . 

4, Extripolating from Research in Other Disciplines 

Re^rkabie benefits seem possible by integrating counseling research 
with the basic research of other disciplines. Goldstein (1966) has suggested 
some possible ways in which social psychology can be applied to research in 
cpuneeling and psychotherapy. Some of the research ^ich shows that the 
expectancy of the experimenter and the expectancy of the subject may be 
related to actual outcome is of considerable relevance to counseling. Ways 
in which both counselor expectancies and client expectancies can be modified 
could be. experimentally investigated. Goldstein suggests other possible 
analogous research problems stemming from social psychology research on 
role compatibility, interpersonal attraction, authoritarianism, cognitive 
dissonance, aiid convergence. 

xwb.r constructive possibility is the suggestion of Zytowski (1966) 



for the use of therapy analogs. He suggests that many practical problems, 
not readily solved in the actual counseling situation, could be investigated 
by setting up experimental analogs in the laboratory. He reviews the evl" 
dence on a number of attempts to simulate counseling In laboratory situational 
For example, instead of working with clients ^d.th real problems of anxiety 
it may be possible to induce artificially some temporary stress (say by 
making false and negative Rorschach interpretations to clients) and then 
experimentally investigate ways of relieving such temporary anxiety. Such 
laboratory investigations may be fruitful in investigating ways to reduce 
hostility and guilt and to increase persistence, frustration tolerance, 
cooperative behavior, and various opinion, attitude and value changes. 

5. Building a Library of Effective Models 

Many client problems are due to behavior deficits. The client simply 
has not learned some of the behavior patterns necessary for coping with his 
particular problem. Some of our research has shoxni that though tape-recorded 
models may be very effective at stimulating learning activities for some 
clients, they are not equally effective with all clients. For example, a 
tape recording which promoted the career exploration of high sjchool boys 
proved ineffective for high school girls. Another video tape proved 
effective for the girls. Counselors could increase their effectiveness if 
they possessed a library of models designed to help various types of students 
learn various kinds of behavior. Extensive research x^ould be needed to 
develop and test these models. 

The Job of building such a library x^ould require the efforts of many 
persons over a long period of time. Research would be needed to establish 
that each model was effective in helping a particular group of students 
learn a particular pattern of behavior. The end result of many such researches 
would be to arm counselors across the country with a library of effective 
learning aids. Problems involved with matching clients and models would 
pose additional problems for research. 



6. Improving the Career Decision Process 

The one area i^ere counselors are reputed to have unchallenged competency 
is in the area of helping clients investigate feasible alternatives. Many 
'clients are unable to make realistic decisions because they have no occupa" 
tional information that seems relevant to them. Much of the traditional 
occupational literature is difficult reading, and many of the answers are 
to questions students never ask. One attempt to solve this problem is to 
provide simulated occupational problem” solving kits (Krumboltz and Sheppard, 
1966) . These kits enable high school students to engage in the problem- 
solving activities of various types of occupations. The problems are 
deliberately designed to be easy, but still realistic, so that students can 
solve the problems and have the feeling of success at working at a given 
occupation. Hopefully, students who experience these problem-solving kits 
will be intrigued by their experience and eager to learn more about the 
occupation they have experienced. Innumerable research questions arise from 
this kind of a project. What is the effect of the difficulty level of the 
problems that are posed? How does the difficulty level interact V7ith the 
ability level of the students? How should the determination be made as to 
which particular occupational kit should be used with which students? Is 
is really advantageous for students to take the kit that they request, or 
to take a kit in an area in which they xiave had no experience? How many 
kits provide the optimum effect? These are some of the questions on ^ich 
we are working now and for which we hope to provide some tentative answers 
in the near future. 

Occupational simulation is just one way to encourage the exploration 
that is necessary for studjnts to make realistic choices. Other ways need 
to be devised and tested to encourage students to explore their orm future 

possibilities. / 

As the result of the work of a competent counselor, clients ought to be 
better able to engage in the total decision making process. We have con- 
sidered a few of the questions involved in the first two steps of the decision 
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making process: posing relevant alternatives, and investigating information 

about these alternatives. Other steps in the decision making process include 
estimating the probability of success in each alternative, relating personal 
values to each course of action, deliberating and weighing all the factors 
which must be considered in eliminating from consideration the least favorable' 
courses of action, formulating a tentative final plan of action, and then 
generalizing this entire decision making process to future probl^s. 

For each of these steps in the decision making process we must ask: How 

can the counselor best help the student? 

How shall the counselor most effectively help students estimate their 
probabilities of success in various alternatives that they have considered? 

The Palo Alto schools under the leadership of H. B. Gelatt have developed 
programed "experience tables" which enable high school students to estimate 
their probability of being accepted into different kinds of colleges on the 
basis of their grade point average to date. The approach appears to be 
effective in giving students a realistic idea of their chances for being 
admitted to each school. Much research will be needed to help students 
estimate their chances of success in various other alternatives they may 
consider. 

How does a counselor help a student arrive at a relative ranking of his 
alternatives and develop a sequence of choices including alternative actions 
in case first choices are not reached? How does the counselor insure that 
this involved decision making process will be learned so that the client will 
use it again as he faces new problems in life? 

7 . Preventing Problems 

Just as responsible dental authorities place high priority on insti- 
tuting floridation to prevent dental problems, so counselors should begin 
to put high priority on ways of preventing problems of inappropriate voca- 
tional choice, inadequate social relationships, unfortunate marriages, wasted 
talent, and inadequate learning. At the moment we really do not know the 
best Ways of preventing problems such as these. Since the answers to 
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problems of this magnitude go far beyond the confines of the counseling Inter* 
view room, some counselors may not recognize their responsibility for helping 
to solve them. Counselors should consider the prevention of such problems 
to be at least as Important as any of the other problems that I have already 
discussed. Preventing problems may Involve administrative arrangements 
within the Institutions In which we work. For example, the grading policies 
In nuiny educational Institutions create far more problems than they solve. 

Another area of concern consists of the dating behavior of adolescents 
which frequently leads them to marry the wrong person for the wrong reasons. 
Very often young people have no opportunity, or little opportunity, to meet 
people vd.th whom they might be most compatible because of the superficialities 
of the Introductory process. Plough we may be dlstalnful of the commercial 
exploitation of so**called "computer dating," the possibilities of this bn 
college canq>uses are being explored by a few of our more courageoua colleagues. 
If a sound scientifically tested system could be developed through which 
yoiing people could be made ax*;are of the names of others x^ith xdiom they might 
be compatible, perhaps otie slight step would have been taken toward Increas- 
ing the future happiness of a significant mimber of persons who would other- 
wise end up at the marital counseling office. 

But the development of a sound preventative program must be based on the 
highest professional ethics, the soundest research designs,, and the most 
noble of human motivations. Who else but counseling psychologists could 
provide all that! 
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